<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
  <meta charset="utf-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  
  <meta name="author" content="Rogerspy">
  <link rel="canonical" href="https://pytorch-zh.gitee.io/torchnn/parameters/">
  <link rel="shortcut icon" href="/pytorch-zh/img/favicon.ico">
  <title>参数 Parameters - Pytorch 中文文档（1.4.0）</title>
  <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700" />

  <link rel="stylesheet" href="../../css/theme.css" />
  <link rel="stylesheet" href="../../css/theme_extra.css" />
  <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/github.min.css" />
  
  <script>
    // Current page data
    var mkdocs_page_name = "\u53c2\u6570 Parameters";
    var mkdocs_page_input_path = "torchnn\\parameters.md";
    var mkdocs_page_url = "/torchnn/parameters/";
  </script>
  
  <script src="../../js/jquery-2.1.1.min.js" defer></script>
  <script src="../../js/modernizr-2.8.3.min.js" defer></script>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script>
  <script>hljs.initHighlightingOnLoad();</script> 
  
</head>

<body class="wy-body-for-nav" role="document">

  <div class="wy-grid-for-nav">

    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side stickynav">
    <div class="wy-side-scroll">
      <div class="wy-side-nav-search">
	    <img src='/pytorch-zh/img/logo.svg'/>
        <a href="../.." class="icon icon-home"> Pytorch 中文文档（1.4.0）</a>
        <div role="search">
  <form id ="rtd-search-form" class="wy-form" action="../../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" title="Type search term here" />
  </form>
</div>
      </div>

      <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
                <ul>
                    <li class="toctree-l1"><a class="reference internal" href="../..">主页</a>
                    </li>
                </ul>
                <ul>
                    <li class="toctree-l1"><a class="reference internal" href="../../get-started/">60分钟快速入门</a>
                    </li>
                </ul>
                <ul>
                    <li class="toctree-l1"><a class="reference internal" href="../../faq/">PyTorch FAQ</a>
                    </li>
                </ul>
                <p class="caption"><span class="caption-text">两种基本结构</span></p>
                <ul class="current">
                    <li class="toctree-l1 current"><a class="reference internal current" href="./">参数 Parameters</a>
    <ul class="current">
    </ul>
                    </li>
                    <li class="toctree-l1"><a class="reference internal" href="#">容器 Containers</a>
    <ul>
                <li class="toctree-l2"><a class="reference internal" href="../module/">Module</a>
                </li>
                <li class="toctree-l2"><a class="reference internal" href="../sequential/">Sequential</a>
                </li>
                <li class="toctree-l2"><a class="reference internal" href="../modulelist/">Modulelist</a>
                </li>
                <li class="toctree-l2"><a class="reference internal" href="../moduledict/">Moduledict</a>
                </li>
                <li class="toctree-l2"><a class="reference internal" href="../parameterlist/">Parameterlist</a>
                </li>
                <li class="toctree-l2"><a class="reference internal" href="../parameterdict/">Parameterdict</a>
                </li>
    </ul>
                    </li>
                </ul>
                <p class="caption"><span class="caption-text">网络层</span></p>
                <ul>
                    <li class="toctree-l1"><a class="reference internal" href="#">卷积层</a>
    <ul>
                <li class="toctree-l2"><a class="reference internal" href="../conv1d/">Conv1d</a>
                </li>
                <li class="toctree-l2"><a class="reference internal" href="../conv2d/">Conv2d</a>
                </li>
                <li class="toctree-l2"><a class="reference internal" href="../conv3d/">Conv3d</a>
                </li>
                <li class="toctree-l2"><a class="reference internal" href="../convtranspose1d/">Convtranspose1d</a>
                </li>
                <li class="toctree-l2"><a class="reference internal" href="../convtranspose2d/">Convtranspose2d</a>
                </li>
                <li class="toctree-l2"><a class="reference internal" href="../convtranspose3d/">Convtranspose3d</a>
                </li>
    </ul>
                    </li>
                </ul>
      </div>
    </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
        <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
        <a href="../..">Pytorch 中文文档（1.4.0）</a>
      </nav>

      
      <div class="wy-nav-content">
        <div class="rst-content">
          <div role="navigation" aria-label="breadcrumbs navigation">
  <ul class="wy-breadcrumbs">
    <li><a href="../..">Docs</a> &raquo;</li>
    
      
        
          <li>两种基本结构 &raquo;</li>
        
      
    
    <li>参数 Parameters</li>
    <li class="wy-breadcrumbs-aside">
      
    </li>
  </ul>
  
  <hr/>
</div>
          <div role="main">
            <div class="section">
              
                <h2 id="parameters">Parameters<a class="headerlink" href="#parameters" title="Permanent link">&para;</a></h2>
<div class='important'><pre><font color='red'>CLASS</font> torch.nn.Parameter(<i>data, requires_grad=False</i>)</pre></div>

<p>一种可视为模块参数的张量。</p>
<p>和 <code>Module</code> 一起使用的时候，<code>Parameter</code> 是具有特殊属性的 <code>Tensor</code> 的子类。当我们把 <code>Parameter</code> 赋值给 <code>Module</code> 作为其属性的时候，它们会自动添加到 <code>Module</code> 的参数列表中，比如在使用 <code>parameters()</code> 方法的时候它们会出现在迭代器中。给 <code>Module</code> 的属性赋值张量的时候不会有这种效果，因为有时候我们需要在模型中缓存一些临时状态，比如 RNN 的最后一个隐状态。如果没有 <code>Parameter</code> 类，临时缓存区也会被注册。</p>
<p><strong>参数</strong></p>
<ul>
<li><em>data（Tensor）</em> ：张量参数；</li>
<li><em>requires_grad（bool, optional）</em>：参数是否需要计算梯度。更多细节查看<a href="https://pytorch.org/docs/stable/notes/autograd.html#excluding-subgraphs">从<code>backward()</code>中剔除子图</a>。默认值：<em>True</em>。</li>
</ul>
<p><strong>示例</strong></p>
<div class="codehilite"><pre><span></span><code><span class="k">class</span> <span class="nc">M</span><span class="p">(</span><span class="n">nn</span><span class="o">.</span><span class="n">Module</span><span class="p">):</span>
    <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">()</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">weights</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Parameter</span><span class="p">(</span><span class="n">torch</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span>

    <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span>
        <span class="k">return</span> <span class="n">x</span> <span class="o">@</span> <span class="bp">self</span><span class="o">.</span><span class="n">weights</span>

<span class="n">m</span><span class="o">=</span><span class="n">M</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">m</span><span class="o">.</span><span class="n">parameters</span><span class="p">()))</span>
</code></pre></div>

<p>输出：</p>
<div class="codehilite"><pre><span></span><code><span class="p">[</span><span class="n">Parameter</span> <span class="n">containing</span><span class="p">:</span>
<span class="n">tensor</span><span class="p">([[</span><span class="o">-</span><span class="mf">0.0439</span><span class="p">,</span>  <span class="mf">0.0548</span><span class="p">],</span>
        <span class="p">[</span><span class="o">-</span><span class="mf">0.0174</span><span class="p">,</span>  <span class="mf">0.0223</span><span class="p">]],</span> <span class="n">requires_grad</span><span class="o">=</span><span class="kc">True</span><span class="p">)]</span>
</code></pre></div>

<p>那么 <code>Parameter</code> 与 <code>Tensor</code> 有什么区别呢？或者说，我们怎么理解 <code>Parameter</code> 这个类呢？</p>
<p>从上面的描述中我们可以得到两个信息：</p>
<ul>
<li><code>Parameter</code> 是 <code>Tensor</code>  的一个子类，这个子类在与 <code>Module</code> 合用的时候会有一些 <code>Tensor</code> 不具有的属性。</li>
<li><code>Parameter</code> 的特殊性表现在，它会自动加入 <code>Module</code> 的参数列表中。</li>
</ul>
<p>我们用例子来说明：</p>
<div class="codehilite"><pre><span></span><code><span class="kn">import</span> <span class="nn">torch.nn</span> <span class="k">as</span> <span class="nn">nn</span>

<span class="k">class</span> <span class="nc">M</span><span class="p">(</span><span class="n">nn</span><span class="o">.</span><span class="n">Module</span><span class="p">):</span>
    <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">()</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">weights1</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Parameter</span><span class="p">(</span><span class="n">torch</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">weights2</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="n">requires_grad</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span>
        <span class="n">x</span> <span class="o">=</span>  <span class="n">x</span> <span class="o">@</span> <span class="bp">self</span><span class="o">.</span><span class="n">weights1</span>
        <span class="k">return</span> <span class="n">x</span> <span class="o">@</span> <span class="bp">self</span><span class="o">.</span><span class="n">weights2</span>

<span class="n">m</span><span class="o">=</span><span class="n">M</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">m</span><span class="o">.</span><span class="n">parameters</span><span class="p">()))</span>
</code></pre></div>

<p>输出：</p>
<div class="codehilite"><pre><span></span><code><span class="p">[</span><span class="n">Parameter</span> <span class="n">containing</span><span class="p">:</span>
<span class="n">tensor</span><span class="p">([[</span><span class="o">-</span><span class="mf">0.0439</span><span class="p">,</span>  <span class="mf">0.0548</span><span class="p">],</span>
        <span class="p">[</span><span class="o">-</span><span class="mf">0.0174</span><span class="p">,</span>  <span class="mf">0.0223</span><span class="p">]],</span> <span class="n">requires_grad</span><span class="o">=</span><span class="kc">True</span><span class="p">)]</span>
</code></pre></div>

<p>我们可以看到，虽然在 <code>__init__</code> 中定义了 2 个<code>Tensor</code> 而且 <code>requires_grad</code> 都设为  <code>True</code>，但是 <code>Module</code> 的参数列表却只包含 1 个元素。也就是说，<code>Tensor</code> 并没有作为 <code>Module</code> 的参数列表。</p>
<p>这样会造成什么影响呢？我们来伪造点数据随便训练一个模型看看：</p>
<div class="codehilite"><pre><span></span><code><span class="kn">import</span> <span class="nn">torch.optim</span> <span class="k">as</span> <span class="nn">optim</span>

<span class="c1"># 定义损失函数和优化器</span>
<span class="n">criterion</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">CrossEntropyLoss</span><span class="p">()</span>
<span class="n">optimizer</span> <span class="o">=</span> <span class="n">optim</span><span class="o">.</span><span class="n">SGD</span><span class="p">(</span><span class="n">m</span><span class="o">.</span><span class="n">parameters</span><span class="p">(),</span> <span class="n">lr</span><span class="o">=</span><span class="mf">0.1</span><span class="p">,</span> <span class="n">momentum</span><span class="o">=</span><span class="mf">0.9</span><span class="p">)</span>

<span class="c1"># 伪造数据</span>
<span class="n">data_x</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="mi">100</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">data_y</span> <span class="o">=</span> <span class="n">data_y</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">ones</span><span class="p">(</span><span class="mi">100</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="n">dtype</span><span class="o">=</span><span class="n">torch</span><span class="o">.</span><span class="n">long</span><span class="p">)</span>
<span class="n">data_y</span><span class="p">[:,</span> <span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="mi">0</span>

<span class="c1"># 训练</span>
<span class="k">for</span> <span class="n">epoch</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">2</span><span class="p">):</span>
    <span class="n">inputs</span><span class="p">,</span> <span class="n">labels</span> <span class="o">=</span> <span class="n">data_x</span><span class="p">,</span> <span class="n">data_y</span>  
    <span class="n">optimizer</span><span class="o">.</span><span class="n">zero_grad</span><span class="p">()</span>  
    <span class="n">outputs</span> <span class="o">=</span> <span class="n">m</span><span class="p">(</span><span class="n">inputs</span><span class="p">)</span>
    <span class="nb">print</span><span class="p">(</span><span class="n">labels</span><span class="o">.</span><span class="n">size</span><span class="p">(),</span> <span class="n">outputs</span><span class="o">.</span><span class="n">size</span><span class="p">())</span>
    <span class="n">loss</span> <span class="o">=</span> <span class="n">criterion</span><span class="p">(</span><span class="n">outputs</span><span class="p">,</span> <span class="n">labels</span><span class="p">)</span>
    <span class="n">loss</span><span class="o">.</span><span class="n">backward</span><span class="p">()</span>
    <span class="n">optimizer</span><span class="o">.</span><span class="n">step</span><span class="p">()</span>
    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;weights1: &#39;</span><span class="p">,</span> <span class="n">m</span><span class="o">.</span><span class="n">weights1</span><span class="p">)</span>
    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;weights2: &#39;</span><span class="p">,</span> <span class="n">m</span><span class="o">.</span><span class="n">weights2</span><span class="p">)</span>
    <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;====================&#39;</span><span class="p">)</span>
</code></pre></div>

<p>输出：</p>
<div class="codehilite"><pre><span></span><code><span class="n">weights1</span><span class="p">:</span>  <span class="n">tensor</span><span class="p">([[</span> <span class="mf">2.2141</span><span class="p">,</span>  <span class="mf">1.7553</span><span class="p">],</span>
        <span class="p">[</span> <span class="mf">1.6770</span><span class="p">,</span> <span class="o">-</span><span class="mf">1.5844</span><span class="p">]],</span> <span class="n">requires_grad</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">weights2</span><span class="p">:</span>  <span class="n">Parameter</span> <span class="n">containing</span><span class="p">:</span>
<span class="n">tensor</span><span class="p">([[</span><span class="o">-</span><span class="mf">0.0439</span><span class="p">,</span>  <span class="mf">0.0372</span><span class="p">],</span>
        <span class="p">[</span><span class="o">-</span><span class="mf">0.0174</span><span class="p">,</span>  <span class="mf">0.0231</span><span class="p">]],</span> <span class="n">requires_grad</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="o">====================</span>
<span class="n">weights1</span><span class="p">:</span>  <span class="n">tensor</span><span class="p">([[</span> <span class="mf">2.2141</span><span class="p">,</span>  <span class="mf">1.7553</span><span class="p">],</span>
        <span class="p">[</span> <span class="mf">1.6770</span><span class="p">,</span> <span class="o">-</span><span class="mf">1.5844</span><span class="p">]],</span> <span class="n">requires_grad</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="n">weights2</span><span class="p">:</span>  <span class="n">Parameter</span> <span class="n">containing</span><span class="p">:</span>
<span class="n">tensor</span><span class="p">([[</span><span class="o">-</span><span class="mf">0.0439</span><span class="p">,</span>  <span class="mf">0.0334</span><span class="p">],</span>
        <span class="p">[</span><span class="o">-</span><span class="mf">0.0174</span><span class="p">,</span>  <span class="mf">0.0154</span><span class="p">]],</span> <span class="n">requires_grad</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</code></pre></div>

<p>我们可以看到，<code>weights1</code> 在训练过程中没有发生任何变化，也就是说，虽然我们把 <code>weights1</code> 设置成了 <code>requires_grad=True</code>，但是在实际训练过程中它并没有得到训练。</p>
<p>也就是说，只有包含在 <code>Module</code> 列表里面的参数才会被训练。这也很容易理解，因为我们在构建优化器的时候，传入的参数就是 <code>m.parameters()</code>，所以优化器优化的就是这部分参数。</p>
<p>当然，我们可以将 <code>weights1</code> 作为参数传递给优化器，但是模型的其他参数也需要优化，要同时优化 <code>weights1</code> 和 <code>weights2</code> 就需要比较复杂的处理了。因此，对于模型需要优化的参数尽量使用 <code>Parameter</code> 而非 <code>Tensor</code>。</p>
              
            </div>
          </div>
          <footer>
  
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
      
        <a href="../module/" class="btn btn-neutral float-right" title="Module">Next <span class="icon icon-circle-arrow-right"></span></a>
      
      
        <a href="../../faq/" class="btn btn-neutral" title="PyTorch FAQ"><span class="icon icon-circle-arrow-left"></span> Previous</a>
      
    </div>
  

  <hr/>

  <div role="contentinfo">
    <!-- Copyright etc -->
    
      <p>©2020 Rogerspy. All rights reserved.</p>
    
  </div>

  Built with <a href="https://www.mkdocs.org/">MkDocs</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
      
        </div>
      </div>

    </section>

  </div>

  <div class="rst-versions" role="note" aria-label="versions">
    <span class="rst-current-version" data-toggle="rst-current-version">
      
      
        <span><a href="../../faq/" style="color: #fcfcfc;">&laquo; Previous</a></span>
      
      
        <span style="margin-left: 15px"><a href="../module/" style="color: #fcfcfc">Next &raquo;</a></span>
      
    </span>
</div>
    <script>var base_url = '../..';</script>
    <script src="../../js/theme.js" defer></script>
      <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML" defer></script>
      <script src="../../search/main.js" defer></script>
    <script defer>
        window.onload = function () {
            SphinxRtdTheme.Navigation.enable(true);
        };
    </script>

</body>
</html>
